United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
I nihil Stall-, l'atint and Trademark Office 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



10/609.593 



09/25/2003 



2292 7590 10/21/2008 

BIRCH STEWART KOLASCH & BIRCH 
PO BOX 747 

FALLS CHURCH, VA 22040-0747 



WANG, JIN CHENG 



PAPER NUMBER 



NOTIFICATION DATE | DELIVERY MODE 
10/21/2008 ELECTRONIC 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 

Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the 

following e-mail address(es): 

mailroom@bskb.com 



PTOL-90A (Rev. 04/07) 



l/ffflrC? nVrliUli Otfff Iff ids y 


Application No. 

10/669,593 


Applicant(s) 

SHIOTA ET AL. 


Examiner 

JIN-CHENG WANG 


Art Unit 

2628 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3_ MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )KI Responsive to communication(s) filed on 01 August 2008 . 
2a )^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) |EI Claim(s) 1-10,16-22,24 and 25 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-10.16-22.24 and 25 is/are rejected. 

7) 0 Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1) ^| Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5 ) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 



PTOL-T26 d (Rev e 08-06r 



Office Action Summary 



Part of Paper No./Mail Date 20081005 



Application/Control Number: 10/669,593 Page 2 

Art Unit: 2628 

DETAILED ACTION 

Response to Amendment 

Applicant's submission filed on 8/1/2008 has been entered. Claims 1-10, 1 1-22, and 24 
have been amended. Claims 11-15 and 23 have been canceled. The claims 1-10, 16-22 and 24-25 
are pending in the application. 

Response to Arguments 

Applicant's arguments filed August 1, 2008 have been fully considered but are moot in 
view of the new ground(s) of rejection of the base claim 1 based on Kahn et al. US 
2004/0004663 Al and Asami U.S. Patent No. 6,882,350 and DeLorme set forth in the present 
Office Action. 

Applicant argues in essence with respect to the claim 1 and similar claims that Kahn and 
Asami do not disclose the claimed additional information. Applicant's arguments are moot in 
view of the new ground(s) of rejection. Moreover, the Kahn, Asami and DeLorme combination 
teaches all claim limitations set forth in the claim 1 . 

As addressed below, Kahn teaches a method for generating an album based on album data 
including at least one image data set, which has been photographed during a trip and which 
has location data representing a photography location attached thereto, comprising the steps 
of: 

Loading the image data into a personal computer {See Paragraph 0044-0045 wherein the 
image data is taken by the digital camera 100 and is loaded to the local host device such as 
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cell phone, personal digital assistant, computer or the like and the personal digital assistant 
or the computer meets the claim limitation of "a personal computer"); 

Forwarding image data from a user terminal of said personal computer to a generation 
support server (e.g., Kahn teaches forwarding/uploading user 's photos from the local host device 
to the image management server of Fig. 4; see Paragraph 0089 and 0097-0098), a user at the 
user terminal being identified by a user ID (user 's GPS location, location ID in paragraph 0094 
and 0097 and user's identity ID of paragraph 0098), where a predetermined reference position 
corresponds to said user ID (e.g., Paragraph 0051 wherein the location information of where the 
user is currently located, the most recently captured images for organizing a plurality of photos. 
The location information may also be a centroid of a given geographic location of interest of 
Kahn meets the claim limitation of "a predetermined reference position " wherein the centroid is 
associated with the user 's location ID and/or the user 's ID; see paragraph 0093); and, 
The generation support server performing the steps of: 
Calculating said distance between the photography location of the image data set and a 
predetermined reference position, based on the location data (e.g., Kahn teaches calculating 
the distance by determining whether the user's GPS location lies within a given geographic 
location of interest to determine the distance between the user 's GPS location and the 
centroid for a given geographic location as performed within an SQL stored procedure; see 
Paragraph 0093); 

Judging whether the distance is over a predetermined threshold value (e.g., the radius of 
a given geographic location as taught in Kahn meets the claim limitation of "a predetermined 
threshold value "; see Paragraph 0093-0094; a list of fulfillers within a given distance as 
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retrieved from the database in the image manager are returned to the local applet and displayed 
to the user as sorted by least distance to the user; see Paragraph 0098-0100); 

Classifying the image data set as one of a vacation album or specific theme album 
according to the result of judgment (Fig. 5F shows the image data set as a vacation album or 
specific theme album such as Friends album or Spring Time album or "Venice Italy" album. The 
"albums" based on location of Paragraph 0098-0100); 

Obtaining from a storage device additional data associated with the classification made 
for the image data set (See Paragraph 0098 the location ID information and metadata 
comprising the latitude and longitude positions are stored as a database record in a database 
table as well as timestamp, user ID are stored in the database. A portion of the data constitute 
the additional information; see Paragraph 0100 the image manager compares the location ID 
against a database of fulfillers and a list of fulfillers within a given distance is returned to the 
local applet which displays the list to the user); and 

Generating the album data according to the result of classification and the image data set 
and the additional data ( "albums " based on location of Paragraph 0089 and Paragraph 98- 
0100; see Paragraph 0100 the image manager compares the location ID against a database of 
fulfillers and a list of fulfillers within a given distance is returned to the local applet which 
displays the list to the user), the additional data including third party images, spot data that 
provides descriptions of areas of interest associated with the photography location (Kahn 
discloses in Paragraph 0098 that the metadata for the images include such fields as the user ID, 
timestamp and location ID that may be assigned a descriptive title that includes information 
indicating geographic location such as "Venice, Italy". The images include the images associated 
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with a particular user ID as well as thirty-party user ID. The location ID includes the longitude 
and latitude decimal values that may be assigned a descriptive title that includes information 
indicating geographic location such as "Venice, Italy", meeting the claimed spot data that 
provides description of areas of interest associated with the photography location). 

Kahn is silent to the claim limitation of displaying together, on a display screen, both 
map data indicating a travel route and thumbnail images in a chronological order. Kahn is silent 
to the additional data including audio data associated with the photography location. 

However, Asami discloses in Figs. 10-12 and 15-19 displaying together on a display 
screen both map data indicating a travel route and thumbnail images in a chronological order 
(See also Asami column 20, lines 34-62, e.g., the map data indicating the sightseeing course of 
the tour or the map data indicating the route connecting the thumbnail icons of photos). 

Although Kahn discloses the claim limitation of obtaining, from a storage device, 
additional data associated with the classification made for the image data set; and generating the 
album data according to the result of classification and the image data set and the additional data, 
Kahn does not explicitly disclose the claim limitation of "a map database" within the claim 
limitation of obtaining, from a storage device and a map database, additional data associated with 
the classification made for the image data set; and generating the album data according to the 
result of classification and the image data set and the additional data. However, Kahn discloses a 
database which stores geographic locations and/or the latitude and longitude information about 
the photos taken at the geographic locations (See Paragraph 0098) and therefore Kahn implicitly 
teaches a map database storing geographic locations and/or the latitude and longitude 
information about the photos taken at the geographic locations. Therefore, Kahn implicitly 
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teaches the claim limitation of obtaining, from a storage device and a map database, additional 
data associated with the classification made for the image data set; and generating the album data 
according to the result of classification and the image data set and the additional data. 

However, Asami further teaches the claim limitation of obtaining, from a storage device 
(e.g., the image position information and image pickup time database) and a map database (e.g., 
the map database 113 of Fig. 9), additional data associated with the classification made for the 
image data set (e.g., the additional information being the position information and time 
information; see Fig. 9; see also column 24, lines 40-67); and generating the album data 
according to the result of classification and the image data set and the additional data (e.g., 
column 24, lines 40-67 generating the album data relating to a selected route according to the 
result of classification based on the image data on a specific route line and a collection of photo 
icons are displayed on as specific route line as controlled by the control movement of the map; 
thumbnail icons are read from memory storage and displayed at any time; changing a route line 
by the control movement changes the corresponding image thumbnail icons being displayed and 
thus the image data set is classified according to a specific route line — or specific theme album 
on the specific route line; see column 25, lines 1-10; see also column 25, lines 55-67). 

Moreover, Asami also discloses the other claim limitations set forth in the claim 1 . For 
example, Asami discloses the claim limitation of judging whether the distance is over a 
predetermined threshold value (the thumbnail icon is positioned based on the longitude and 
latitude information and is obtained by matching the position information and thus Asami 
implicitly teaches a match distance between the position coordinates of the travel route line and 
the position coordinates of the photo icons. Asami implicitly discloses that a match distance 
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between the image pickup time and the pickup time of the photos. Moreover, the thumbnail icons 
are obtained based on the judgment that the coordinates of the photos are in match with the 
coordinates of the travel route line and a predetermined number of photo icons possessing the 
pickup time data are extracted; see column 22, lines 5-35) and classifying the image data set as 
one of a vacation album or specific theme album according to the result of judgment (e.g., at 
column 24, lines 40-67 Asami teaches generating the album data relating to a selected route 
according to the result of classification based on the image data on a specific route line and a 
collection of photo icons are displayed on as specific route line as controlled by the control 
movement of the map; thumbnail icons are read from memory storage and displayed at any time; 
changing a route line by the control movement changes the corresponding image thumbnail 
icons being displayed and thus the image data set is classified according to a specific route line- 
-or specific theme album on the specific route line; see column 25, lines 1-10) , the additional 
data including third party images, spot data that provides descriptions of areas of interest 
associated with the photography location (Asami discloses in Figs. 16-19 and column 26, lines 
43-50 associating the thumbnail icons with the spot data- — text comment — that provides 
descriptions of areas of interest associated with the photography location. Asami discloses in 
column 19, lines 5-20 that a thumbnail ID is delivered to make the icon a thumbnail icon and if a 
user ID is delivered newly, then a message representing delivery of a user ID such as "A user ID 
is delivered" may be presented to the user". In the same manner as Kahn, Asami discloses the 
user ID and the thumbnail ID are associated with the images. Asami thus teaches the thumbnail 
icons are not limited to a particular user, i.e., the thumbnail icons are associated with the images 
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by a particular user as well as the third party user. Therefore, Asami teaches the thirty party 
images). 

It would have been obvious to one of the ordinary skill in the art at the time the invention 
was made to have incorporated Asami into Kahn's method and apparatus for generating an 
album because Kahn's photos are location dependent in a trip and are ordered in chronological 
order in a trip and Kahn's photos thus can be displayed according to the teaching of Asami (Figs. 
15-19 and 10-12 of Asami) as Asami's photos are also locations dependent and are ordered 
chronologically for a trip. 

One of the ordinary skill in the art would have been motivated to incorporate Asami to 
display the photo icons along the travel route (See also Asami column 20, lines 34-62, e.g., the 
map data indicating the sightseeing course of the tour or the map data indicating the route 
connecting the thumbnail icons of photos; See also Asami Figs. 10-12 and 15-19). 

Although Asami and Kahn do not explicitly teach that the additional data includes the 
audio data associated with the photography location, DeLorme et al. teach the claim limitation. 
For example, in column 10, lines 50-64 and column 13, lines 30-55, DeLorme teaches the POI 
on a travel route provides overlays of the CARPS database for display over the electronic maps 
on the computer display. The database includes the travel information selected from the group 
consisting of (third party) graphics, photos, videos, animations, audio information and text 
information about transportation routes and waypoints of the electronic maps and points of 
interest of the CARPS database. DeLorme discloses the POIs or geographical sites or locations 
for which multimedia information (third party images) are available in the CARPS database and 
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POIs can be represented in both digital and print media cartography described by standard 
geographic coordinates such as latitude and longitude. DeLorme discloses at column 14, lines 
15-35 that the multimedia presentation includes video pictures of the lake, local events, 
attractions and recreational opportunities, related text or audio narrative, etc. 

It would have been obvious to one of the ordinary skill in the art at the time the invention 
was made to have incorporated DeLorme into Asami and Kahn's method and apparatus for 
generating an album wherein a particular point of interest on the travel route on a map can be 
arranged with the additional data including the audio data of DeLorme such that the additional 
data may be displayed upon the user's selection (DeLorme column 14, lines 15-35). 

One of the ordinary skill in the art would have been motivated to incorporate DeLorme to 
display the multimedia data including the audio data along with the photos on a particular point 
of interest along the travel route (See also Asami column 20, lines 34-62, e.g., the map data 
indicating the sightseeing course of the tour or the map data indicating the route connecting the 
thumbnail icons of photos; See also Asami Figs. 10-12 and 15-19, DeLorme column 13-14) such 
that the user's interaction with the multi-media about a particular geographic location is available 
upon request (column 14, lines 50-67). 

Re claims 3-5, 8-10, 13-15, 18-20, Kahn teaches in Paragraph 0098-0100 of obtaining 
related data, which is related to the photography location of the at least one image data set for 
which the distance is over the predetermined threshold value, based on the location data attached 
thereto, from a related data storage means which stores a plurality of related data sets and 
generating album data which includes the related data (See Paragraph 0098 the location ID 
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information and metadata comprising the latitude and longitude positions are stored as a 
database record in a database table as well as timestamp, user ID are stored in the database. A 
portion of the data constitute the additional information; see Paragraph 0100 the image 
manager compares the location ID against a database of fulfillers and a list of fulfillers within a 
given distance is returned to the local applet which displays the list to the user). 

Re Claims 21-24: Kahn further discloses that the predetermined reference position is 
registered as user data (See Paragraph 0098 the location ID information and metadata comprising 
the latitude and longitude positions are stored as a database record in a database table as well as 
timestamp, user ID are stored in the database. A portion of the data constitutes the additional 
information; see Paragraph 0100 the image manager compares the location ID against a database 
of fulfillers and a list of fulfillers within a given distance is returned to the local applet which 
displays the list to the user). 



Claim Rejections - 35 USC § 112 
Claims 16-20 and 24 are rejected under 35 U.S.C. 1 12, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. Claim 16 recites "A computer readable medium". The 
claimed computer readable medium is not limited to the tangible embodiment, but includes an 
intangible embodiment such as transmission means or internet downloads (see Page 48 of 
applicant's originally filed specification). A data structure in an internet download or 
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transmission medium or signal has been included in the embodiment. It is thus not clear whether 
an apparatus claim or a process claim is claimed in the claim 16. Moreover, the body of the claim 
16 recites process steps while an apparatus is recited in the preamble. Clarification is required. 
The claims 17-20 and 24 depend upon the claim 16 and are subject to the same rationale of 
rejection set forth in the claim 16. 

Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

Claims 16-20 and 24: 

Claim 16 recites "A computer readable medium". The claimed computer readable 
medium is not limited to the tangible embodiment, but includes an intangible embodiment such 
as transmission means or internet downloads (See Page 48 of applicant's specification) . A data 
structure in an internet download or transmission medium or signal has been included in the 
embodiment. Therefore, the claim 16 is non-statutory. 

The claims 17-20 and 24 are subject to the same rationale of rejection set forth in the 
claim 18. 



Computer-Related Nonstatutory Subject Matter 
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Descriptive material can be characterized as either "functional descriptive material" or 
"nonfunctional descriptive material." In this context, "functional descriptive material" consists of 
data structures and computer programs which impart functionality when employed as a computer 
component . (The definition of "data structure" is "a physical or logical relationship among data 
elements, designed to support specific data manipulation functions." The New IEEE Standard 
Dictionary of Electrical and Electronics Terms 308 (5th ed. 1993).) "Nonfunctional descriptive 
material" includes but is not limited to music, literary works and a compilation or mere 
arrangement of data. 

Both types of "descriptive material" are nonstatutory when claimed as descriptive material 
per se. Warmerdam, 33 F.3d at 1360, 31 USPQ2d at 1759. See Warmerdam, 33 F.3d at 1361, 31 
USPQ2d at 1760 (claim to a data structure per se held nonstatutory). 

When nonfunctional descriptive material is recorded on some computer-readable medium, in 
a computer or on an electromagnetic carrier signal, it is not statutory since no requisite 
functionality is present to satisfy the practical application requirement. Merely claiming 
nonfunctional descriptive material, i.e., abstract ideas, stored in a computer-readable medium, in 
a computer, on an electromagnetic carrier signal does not make it statutory. See Diehr, 450 U.S. 
at 185-86, 209 USPQ at 8 ( noting that the claims for an algorithm in Benson were unpatentable 
as abstract ideas because "[t|he sole practical application of the algorithm was in connection with 
the programming of a general purpose computer." ). Such a result would exalt form over 
substance. In re Sarkar, 588 F.2d 1330, 1333, 200 USPQ 132, 137 (CCPA 1978) ("[E]ach 
invention must be evaluated as claimed; yet semantogenic considerations preclude a 
determination based solely on words appearing in the claims. In the final analysis under § 101, 
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the claimed invention, as a whole, must be evaluated for what it is.") (quoted with approval in 
Abele, 684 F.2d at 907, 214 USPQ at 687). See also In re Johnson, 589 F.2d 1070, 1077, 200 
USPQ 199, 206 (CCPA 1978) ("form of the claim is often an exercise in drafting"). Thus, 
nonstatutory music is not a computer component and it does not become statutory by merely 
recording it on a compact disk. Protection for this type of work is provided under the copyright 
law. 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1-10, 16-22 and 24-25 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Kahn et al. US 2004/0004663 Al (hereinafter Kahn) in view of Asami U.S. Patent No. 
6,882,350 (hereinafter Asami) and DeLorme et al. U.S. Patent No. 5,802,492 (hereinafter 
DeLorme). 



Re Claims 1, 6, 16 and 25: 
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Kami teaches a method for generating an album based on album data including at least one 
image data set, which has been photographed during a trip and which has location data 
representing a photography location attached thereto, comprising the steps of: 

Loading the image data into a personal computer (See Paragraph 0044-0045 wherein the 
image data is taken by the digital camera 100 and is loaded to the local host device such as 
cell phone, personal digital assistant, computer or the like and the personal digital assistant 
or the computer meets the claim limitation of "a personal computer"); 

Forwarding image data from a user terminal of said personal computer to a generation 
support server (e.g., Kahn teaches forwarding/uploading user 's photos from the local host device 
to the image management server of Fig. 4; see Paragraph 0089 and 0097-0098), a user at the 
user terminal being identified by a user ID (user 's GPS location, location ID in paragraph 0094 
and 0097 and user's identity ID of paragraph 0098), where a predetermined reference position 
corresponds to said user ID (e.g., Paragraph 0051 wherein the location information of where the 
user is currently located, the most recently captured images for organizing a plurality of photos. 
The location information may also be a centroid of a given geographic location of interest of 
Kahn meets the claim limitation of "a predetermined reference position " wherein the centroid is 
associated with the user 's location ID and/or the user 's ID; see paragraph 0093); and, 
The generation support server performing the steps of: 
Calculating said distance between the photography location of the image data set and a 
predetermined reference position, based on the location data (e.g., Kahn teaches calculating 
the distance by determining whether the user's GPS location lies within a given geographic 
location of interest to determine the distance between the user 's GPS location and the 
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centroid for a given geographic location as performed within an SQL stored procedure; see 
Paragraph 0093); 

Judging whether the distance is over a predetermined threshold value (e.g., the radius of 
a given geographic location as taught in Kahn meets the claim limitation of "a predetermined 
threshold value "; see Paragraph 0093-0094; a list of fulfillers within a given distance as 
retrieved from the database in the image manager are returned to the local applet and displayed 
to the user as sorted by least distance to the user; see Paragraph 0098-0100); 

Classifying the image data set as one of a vacation album or specific theme album 
according to the result of judgment (Fig. 5 F shows the image data set as a vacation album or 
specific theme album such as Friends album or Spring Time album or "Venice Italy" album. The 
"albums" based on location of Paragraph 0098-0100); 

Obtaining from a storage device additional data associated with the classification made 
for the image data set (See Paragraph 0098 the location ID information and metadata 
comprising the latitude and longitude positions are stored as a database record in a database 
table as well as timestamp, user ID are stored in the database. A portion of the data constitute 
the additional information; see Paragraph 0100 the image manager compares the location ID 
against a database of fulfillers and a list of fulfillers within a given distance is returned to the 
local applet which displays the list to the user); and 

Generating the album data according to the result of classification and the image data set 
and the additional data ( "albums " based on location of Paragraph 0089 and Paragraph 98- 
0100; see Paragraph 0100 the image manager compares the location ID against a database of 
fulfillers and a list of fulfillers within a given distance is returned to the local applet which 
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displays the list to the user), the additional data including third party images, spot data that 
provides descriptions of areas of interest associated with the photography location (Kahn 
discloses in Paragraph 0098 that the metadata for the images include such fields as the user ID, 
timestamp and location ID that may be assigned a descriptive title that includes information 
indicating geographic location such as "Venice, Italy". The images include the images associated 
with a particular user ID as well as thirty-party user ID. The location ID includes the longitude 
and latitude decimal values that may be assigned a descriptive title that includes information 
indicating geographic location such as "Venice, Italy", meeting the claimed spot data that 
provides description of areas of interest associated with the photography location). 

Kahn is silent to the claim limitation of displaying together, on a display screen, both 
map data indicating a travel route and thumbnail images in a chronological order. Kahn is silent 
to the additional data including audio data associated with the photography location. 

However, Asami discloses in Figs. 10-12 and 15-19 displaying together on a display 
screen both map data indicating a travel route and thumbnail images in a chronological order 
(See also Asami column 20, lines 34-62, e.g., the map data indicating the sightseeing course of 
the tour or the map data indicating the route connecting the thumbnail icons of photos). 

Although Kahn discloses the claim limitation of obtaining, from a storage device, 
additional data associated with the classification made for the image data set; and generating the 
album data according to the result of classification and the image data set and the additional data, 
Kahn does not explicitly disclose the claim limitation of "a map database" within the claim 
limitation of obtaining, from a storage device and a map database, additional data associated with 
the classification made for the image data set; and generating the album data according to the 
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result of classification and the image data set and the additional data. However, Kahn discloses a 
database which stores geographic locations and/or the latitude and longitude information about 
the photos taken at the geographic locations (See Paragraph 0098) and therefore Kahn implicitly 
teaches a map database storing geographic locations and/or the latitude and longitude 
information about the photos taken at the geographic locations. Therefore, Kahn implicitly 
teaches the claim limitation of obtaining, from a storage device and a map database, additional 
data associated with the classification made for the image data set; and generating the album data 
according to the result of classification and the image data set and the additional data. 

However, Asami further teaches the claim limitation of obtaining, from a storage device 
(e.g., the image position information and image pickup time database) and a map database (e.g., 
the map database 113 of Fig. 9), additional data associated with the classification made for the 
image data set (e.g., the additional information being the position information and time 
information; see Fig. 9; see also column 24, lines 40-67); and generating the album data 
according to the result of classification and the image data set and the additional data (e.g., 
column 24, lines 40-67 generating the album data relating to a selected route according to the 
result of classification based on the image data on a specific route line and a collection of photo 
icons are displayed on as specific route line as controlled by the control movement of the map; 
thumbnail icons are read from memory storage and displayed at any time; changing a route line 
by the control movement changes the corresponding image thumbnail icons being displayed and 
thus the image data set is classified according to a specific route line — or specific theme album 
on the specific route line; see column 25, lines 1-10; see also column 25, lines 55-67). 
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Moreover, Asami also discloses the other claim limitations set forth in the claim 1 . For 
example, Asami discloses the claim limitation of judging whether the distance is over a 
predetermined threshold value (the thumbnail icon is positioned based on the longitude and 
latitude information and is obtained by matching the position information and thus Asami 
implicitly teaches a match distance between the position coordinates of the travel route line and 
the position coordinates of the photo icons. Asami implicitly discloses that a match distance 
between the image pickup time and the pickup time of the photos. Moreover, the thumbnail icons 
are obtained based on the judgment that the coordinates of the photos are in match with the 
coordinates of the travel route line and a predetermined number of photo icons possessing the 
pickup time data are extracted; see column 22, lines 5-35) and classifying the image data set as 
one of a vacation album or specific theme album according to the result of judgment (e.g., at 
column 24, lines 40-67 Asami teaches generating the album data relating to a selected route 
according to the result of classification based on the image data on a specific route line and a 
collection of photo icons are displayed on as specific route line as controlled by the control 
movement of the map; thumbnail icons are read from memory storage and displayed at any time; 
changing a route line by the control movement changes the corresponding image thumbnail 
icons being displayed and thus the image data set is classified according to a specific route line— 
-or specific theme album on the specific route line; see column 25, lines 1-10) , the additional 
data including third party images, spot data that provides descriptions of areas of interest 
associated with the photography location (Asami discloses in Figs. 16-19 and column 26, lines 
43-50 associating the thumbnail icons with the spot data — text comment — that provides 
descriptions of areas of interest associated with the photography location. Asami discloses in 
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column 19, lines 5-20 that a thumbnail ID is delivered to make the icon a thumbnail icon and if a 
user ID is delivered newly, then a message representing delivery of a user ID such as "A user ID 
is delivered" may be presented to the user". In the same manner as Kahn, Asami discloses the 
user ID and the thumbnail ID are associated with the images. Asami thus teaches the thumbnail 
icons are not limited to a particular user, i.e., the thumbnail icons are associated with the images 
by a particular user as well as the third party user. Therefore, Asami teaches the thirty party 
images). 

It would have been obvious to one of the ordinary skill in the art at the time the invention 
was made to have incorporated Asami into Kahn's method and apparatus for generating an 
album because Kahn's photos are location dependent in a trip and are ordered in chronological 
order in a trip and Kahn's photos thus can be displayed according to the teaching of Asami (Figs. 
15-19 and 10-12 of Asami) as Asami's photos are also locations dependent and are ordered 
chronologically for a trip. 

One of the ordinary skill in the art would have been motivated to incorporate Asami to 
display the photo icons along the travel route (See also Asami column 20, lines 34-62, e.g., the 
map data indicating the sightseeing course of the tour or the map data indicating the route 
connecting the thumbnail icons of photos; See also Asami Figs. 10-12 and 15-19). 

Although Asami and Kahn do not explicitly teach that the additional data includes the 
audio data associated with the photography location, DeLorme et al. teach the claim limitation. 
For example, in column 10, lines 50-64 and column 13, lines 30-55, DeLorme teaches the POI 
on a travel route provides overlays of the CARPS database for display over the electronic maps 
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on the computer display. The database includes the travel information selected from the group 
consisting of (third party) graphics, photos, videos, animations, audio information and text 
information about transportation routes and waypoints of the electronic maps and points of 
interest of the CARPS database. DeLorme discloses the POIs or geographical sites or locations 
for which multimedia information (third party images) are available in the CARPS database and 
POIs can be represented in both digital and print media cartography described by standard 
geographic coordinates such as latitude and longitude. DeLorme discloses at column 14, lines 
15-35 that the multimedia presentation includes video pictures of the lake, local events, 
attractions and recreational opportunities, related text or audio narrative, etc. 

It would have been obvious to one of the ordinary skill in the art at the time the invention 
was made to have incorporated DeLorme into Asami and Kahn's method and apparatus for 
generating an album wherein a particular point of interest on the travel route on a map can be 
arranged with the additional data including the audio data of DeLorme such that the additional 
data may be displayed upon the user's selection (DeLorme column 14, lines 15-35). 

One of the ordinary skill in the art would have been motivated to incorporate DeLorme to 
display the multimedia data including the audio data along with the photos on a particular point 
of interest along the travel route (See also Asami column 20, lines 34-62, e.g., the map data 
indicating the sightseeing course of the tour or the map data indicating the route connecting the 
thumbnail icons of photos; See also Asami Figs. 10-12 and 15-19, DeLorme column 13-14) such 
that the user's interaction with the multi-media about a particular geographic location is available 
upon request (column 14, lines 50-67). 
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Re claims 3-5, 8-10, 18-20, Kahn teaches in Paragraph 0098-0100 of obtaining related 
data, which is related to the photography location of the at least one image data set for which the 
distance is over the predetermined threshold value, based on the location data attached thereto, 
from a related data storage means which stores a plurality of related data sets and generating 
album data which includes the related data (See Paragraph 0098 the location ID information and 
metadata comprising the latitude and longitude positions are stored as a database record in a 
database table as well as timestamp, user ID are stored in the database. A portion of the data 
constitute the additional information; see Paragraph 0100 the image manager compares the 
location ID against a database offulfillers and a list of fulfillers within a given distance is 
returned to the local applet which displays the list to the user). 

Re Claims 21-22 and 24: Kahn further discloses that the predetermined reference position 
is registered as user data (See Paragraph 0098 the location ID information and metadata 
comprising the latitude and longitude positions are stored as a database record in a database table 
as well as timestamp, user ID are stored in the database. A portion of the data constitutes the 
additional information; see Paragraph 0100 the image manager compares the location ID against 
a database of fulfillers and a list of fulfillers within a given distance is returned to the local applet 
which displays the list to the user). 

Re claims 2, 7 and 17, Kahn and Asami further teach the claim limitation of generating 
travel route data, which represents a route taken during the trip (e.g., at Asami column 24, lines 
40-67 Asami teaches generating the album data relating to a selected route according to the 
result of classification based on the image data on a specific route line and a collection of photo 
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icons are displayed on as specific route line as controlled by the control movement of the map; 
thumbnail icons are read from memory storage and displayed at any time; changing a route line 
by the control movement changes the corresponding image thumbnail icons being displayed and 
thus the image data set is classified according to a specific route line — or specific theme album 
on the specific route line; see column 25, lines 1-10; see also column 25, lines 55-67), based on 
the location data attached to the at least one image data set for which the distance is over the 
predetermined threshold value (e.g., at column 24, lines 40-67 Asami teaches generating the 
album data relating to a selected route according to the result of classification based on the 
image data on a specific route line and a collection of photo icons are displayed on as specific 
route line as controlled by the control movement of the map; thumbnail icons are read from 
memory storage and displayed at any time; changing a route line by the control movement 
changes the corresponding image thumbnail icons being displayed and thus the image data set is 
classified according to a specific route line — or specific theme album on the specific route line 
and the image data are matched and searched so as to obtain the relevant photo icons for 
displaying along the travel route wherein the searching process requires the distance between 
the coordinates of the route and the coordinates of the photos be calculated and compared and 
the photos satisfying the distance criteria are obtained and displayed; see column 25, lines 1- 
10); obtaining a map data set that contains the route from a map database which stores a plurality 
of map data sets (Kahn discloses a database which stores geographic locations and/or the 
latitude and longitude information about the photos taken at the geographic locations (See Kahn 
Paragraph 0098 wherein the geographic locations constitutes a map data set that forms a route 
for the photos taken during a trip) and therefore Kahn implicitly teaches a map database storing 



Application/Control Number: 10/669,593 Page 23 

Art Unit: 2628 

geographic locations and/or the latitude and longitude information about the photos taken at the 
geographic locations. Therefore, Kahn implicitly teaches the claim limitation of obtaining, from 
a storage device and a map database, additional data associated with the classification made for 
the image data set; and generating the album data according to the result of classification and 
the image data set and the additional data), based on the travel route data; generating 
photography data which represents that the image data set was obtained at the photography 
location along the route (See Kahn Paragraph 0098 wherein the geographic locations constitutes 
a map data set that forms a route for the photos taken during a trip), attaching the photography 
data to the map data set and generating album data including the map data set in which the 
photography data is correlated with the image data set (See Kahn Paragraph 0098 the location 
ID information and metadata comprising the latitude and longitude positions are stored as a 
database record in a database table as well as timestamp, user ID are stored in the database. A 
portion of the data constitute the additional information; see Kahn Paragraph 0100 the image 
manager compares the location ID against a database offulfillers and a list of fulfillers within a 
given distance is returned to the local applet which displays the list to the user). 

It would have been obvious to one of the ordinary skill in the art at the time the invention 
was made to have incorporated Asami into Kahn's method and apparatus for generating an 
album because Kahn's photos are location dependent in a trip and are ordered in chronological 
order in a trip and Kahn's photos thus can be displayed according to the teaching of Asami (Figs. 
15-19 and 10-12 of Asami) as Asami's photos are also locations dependent and are ordered 
chronologically for a trip. 
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One of the ordinary skill in the art would have been motivated to incorporate Asami to 
display the photo icons along the travel route (See also Asami column 20, lines 34-62, e.g., the 
map data indicating the sightseeing course of the tour or the map data indicating the route 
connecting the thumbnail icons of photos; See also Asami Figs. 10-12 and 15-19). 



Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (571) 272-7665. 
The examiner can normally be reached on 8:00 - 6:30 (Mon-Thu). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee Tung can be reached on (571) 272-7794. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



/Jin-Cheng Wang/ 

Primary Examiner, Art Unit 2628 



